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Abstract 

The prisoner's dilemma (PD) game is a simple model for understanding cooperative patterns 
in complex systems consisting of selfish individuals. Here, we study a PD game problem in scale- 
free networks containing hierarchically organized modules and controllable shortcuts connecting 
separated hubs. We find that cooperator clusters exhibit a percolation transition in the parameter 
space {p,b), where p is the occupation probability of shortcuts and b is the temptation payoff in 
the PD game. The cluster size distribution follows a power law at the transition point. Such a 
critical behavior, resulting from the combined effect of stochastic processes in the PD game and 
the heterogeneous structure of complex networks, illustrates the diversity of social relationships 
and the self-organization of cooperator communities in real-world systems. 

PACS numbers: 89.75.Hc, 89.75.Da, 89.75.Fb 
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Social interactions between individuals are often cooperative or competitive, and certain 
emerging patterns in these interactions can be recognized. In particular, an understanding 
of social dilemmas where the optimal strategy for the whole society is not the same as that 
for each individual has been an attractive topic of interdisciplinary research. The prisoner's 
dilemma (PD) game has been used as a basic model in understanding the patterns that 
emerge in the case of social dilemmas [l, 2|. In the PD game, which was originally designed 
as a two-player game, a player earns a larger payoff by unilateral defection than by mutual 
cooperation. Therefore, even if both players attain nothing when they defect, the optimal 
choice would be defection. On the other hand, in real systems, people are cooperative and 
altruistic. Thus, it will be interesting to resolve such a paradox by using a simple model. 

A social system is more accurately described by a network of players 3|. If players in 
a network change their strategies according to evolutionary dynamics (i.e., by imitating 
successful neighbors), we often observe mutual cooperation, and the system exhibits diverse 
patterns {2], 0] . These diverse patterns result from the collective dynamics of the players' 
interactions with each other p]. While such diverse patterns are present and probably 
functional in social relationships, their diversity has not been clearly understood yet. In this 
Letter, we investigate the origin of the formation of such diverse patterns by using the PD 
game. 

The enhanced cooperation and emerging patterns in the PD game were first clarified in 
the Euclidean space [GJ . Recently, the PD game was extended to complex heterogeneous 
networks such as scale-free (SF) networks, in which the number of the nearest neighbors 
of a node (which is known as the degree of the node) follows a power law. In random SF 
networks such as the Barabasi and Albert model [3], the mean distance between two nodes 
scales logarithmically with the system size. In such small-world SF networks, the density 
of cooperators is significantly higher js] than that in the case of other networks, including 
those in the Euclidean space. Once cooperation is stabilized at the hubs at an early stage 
of dynamics, cooperation spreads to nodes with smaller degrees [8|, 19|. Thus, the hub plays 
a crucial role in spreading cooperation. 

Many SF networks in the real world are not as random as that in the Barabasi and Albert 
model; they contain modular structure. Moreover, the modular structure is hierarchically 



organized 



10| . In such modular SF networks, hubs are separated from each other, and the 



mean distance between two nodes often scales in a power-law manner with the system size 
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Such networks are called large- world or fractal networks. Cooperation would in such 



networks is less than that in random SF networks, because the large distance between hubs 



generally reduces cooperation 



l9|. 



Social networks in the real world are often at the boundary between small-world and large- 
world networks jl2|. Thus, in this Letter, we consider the PD game in artificial networks 
in which the number of edges between the separated hubs is controlled by the occupation 
probability p and examine the patterns in the interactions between the cooperators as the 
network transforms from a large- world to a small-world networks as p increases. We find that 
clusters composed of cooperators undergo a percolation transition in the parameter space 
of (p, fe), where b is the temptation payoff in the PD game. Interestingly, the percolation 
transition occurs either continuously or discontinuously as p is increased for a fixed 6 or 6 is 
decreased for a fixed p. Therefore, there exists a tri critical-like point {pt, bt) such that for a 
fixed b < bt {b > bt) oi p < pt {p > pt), the giant cluster of cooperators grows continuously 
(discontinuously) (Fig.[T](a)). The phase diagram is shown in Fig. [l](b). Furthermore, the size 
distribution of the cooperator clusters exhibits a power-law behavior near the percolation 
threshold as long as p < pt- This emerging pattern suggests that the cooperators self- 
organize their clusters to attain a critical state, which enhances the diversity within the 
system. 

On the basis of previous papers we define the payoff matrix by 



(1) 



A row player selects one of two states, either cooperation (C) or defection (D), so does a 
column player. Each entry in Eq. ([1]) represents the payoff that the row player obtains. If 
the two players start with mutual cooperation, both players are tempted to defect in order 
to obtain a larger payoff (temptation) i.e., 6 > 1. If both players act selfishly and choose to 
defect, both obtain no payoff; this is the unique Nash equilibrium of the game. However, 
the individual rewards for both the players could be higher if they mutually cooperate. 

We examine the evolutionary PD game for several types of complex networks with equal 
initial densities of C and D. At each time step (or round), each player i interacts with all of its 
ki neighbors; here, ki is the degree of the node i. Player z's payoff in one round Pj is the sum 
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FIG. 1: (Color online) (a) Giant cluster size in the parameter space of (p, 6) for the hierarchical 
network with system size = 10, 924. Data are averaged over 100 network configurations, (b) 
Susceptibility under the same condition as that in the case of (a). The solid and dotted curves 
are the loci along the peaks of susceptibility: the peaks represent continuous and discontinuous 
percolation transitions, respectively. 



of all the payoffs earned by playing against the ki neighbors. Player i updates its strategy 
according to the following rule |8|: A neighbor j of player i is chosen with equal probability 
1/ki. Then, if Pj > Pi, i copies j's strategy with probability {Pj — Pi) j [6 max (fcj, /cj)]. The 
denominator normalizes the probability such that the probability is between and 1. On 
the other hand, if Pj < Pi, i does not change its strategy. The rule for updating strategies is 
synchronously applied by all the players. This procedure is repeated in subsequent rounds. 

In each round, a player chooses either C or D. However, in long-term dynamics, players 
are categorized as joj: a permanent cooperator, a permanent defector, or an unstable player 
that continuously changes its state between C and D. We numerically simulate the PD game 
until the density of the cooperators does not change with time. We simulate the PD game 
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FIG. 2: (Color online) Density of cooperators as a function of temptation payoff b for the two 
extreme cases, p = and p = 1 in the hierarchical model. 



for up to 2 X 10"* rounds after a steady state is reached for each players' configuration at 
initial. A permanent cooperator (defector) is defined as the node that chooses only C (D) 
for the last 10^ rounds. The rest of the nodes that do not satisfy this condition are regarded 
as unstable players. The density of the permanent cooperators is denoted by pc- 

The PD game is played on the so-called hierarchical network, introduced in [l^. This 
network is constructed by repeating a simple structural mapping as follows: We begin with 
an edge, which is replaced with a diamond in the next iteration step. A horizontal edge 
between two nodes at the diagonal positions is present with probability p. This process is 
repeated for each edge in the diamond (except the horizontal bond) in subsequent iterations 
until the obtained network attains the desired system size. This network is scale-free and has 
a degree exponent of 3. When p = 0, the network is a large- world network and the diameter, 
i.e., the largest distance between any two nodes in the system, increases according to a power 
law. When p = 1, the network is a small- world network and the diameter is proportional to 
In A^, where A^ is the system size. The phase transition pattern of the Ising model based on 
this hierarchical model indicates that as p increases and crosses p = p* = 0.494, the network 
undergoes a transition from being a large-world network to being a small-world network 
jl^ . The transition behavior is different across p*. In the large-world and small- world, the 
transition is of the Ising type and the inverted Berezinskii-Kosteritz-Thouless, respectively. 

The density of the permanent cooperators pc is shown as a function of the temptation 
payoff b for the two extreme cases corresponding to p = and 1 in Fig. |2j While the mean 
field theory predicts that no cooperator is present in the PD game for 6 > 1, when p = 1, 



FIG. 3: (Color online) A snapshot of permanent cooperators (red, dark), permanent defectors 
(white), and unstable players (cyan, gray) in the hierarchical network with p = 0.15, b = 2.3 and 
N = 684 after 20,000 rounds in a steady state. 



however, pc is almost equal to 1 for b values up to be, beyond which pc drops suddenly. When 
p = 0, pc is relatively small even when b = 1, and it decays continuously with increasing 
b. The discontinuity at 6 = 2 is because of the deterministic structure of the hierarchical 
network; however, such behavior does not appear in other real- world networks. In fact, 
such different types of behaviors of pc with respect to b were also observed in real-world 
networks, in particular, an email network and a social network by using the "Pretty-Good- 



Privacy" program 15[. In the former (latter) case, the cooperator density decreases smoothly 



(drastically). Furthermore, it is noteworthy that for all b, pc ai p = 1 is larger than pc at 
p = 0. This indicates that the shortcuts connecting the hubs play an important role in 
enhancing cooperation js, The cooperation between influential individuals enhances the 
overall cooperation in the society. Let us now proceed to evaluate the case in which p 
increases from to 1. 

A snapshot of the states of the players in the hierarchical network with p = 0.15 b = 
2.3 and = 684 is shown in Fig. |3l this network represents the large-world network. 
The permanent cooperators tend to be located around or at hubs, and may impact other 
cooperators. This is because once cooperators form a cluster around a hub, then the hub 
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becomes stable and is protectd against any invasion by defectors. However, when defectors 
gather around the hub, the hub may cease to be stable because mutual defectors get no 
payoff. Therefore, permanent defectors are located at nodes with small degree, while the 
unstable players are in between nodes with small degrees and those with large degrees. 
We remark that cooperators are not always located at or around hubs as they were in the 
snapshot, because the formation of cooperator clusters is determined stochastically and it 
generally depends on the fluctuations in cooperator densities. The degree of the hubs and 
the dynamic stochastic process of the PD game result in the formation of cooperator clusters 
with a wide range of sizes. 

In contrast, in the small- world network with a large p, for example, p = 1, permanent 
cooperators are always located at hubs and mutual cooperation occurs on a global scale. 
This behavior occurs even when a hub and its neighbors are mostly defectors at an early 
stage; the hub eventually cooperates with another hub. 

The size G{p, b) of the largest cluster of permanent cooperators is shown as a function of 
p and b in Fig. [T|^a). Here G{p, b) is considered to be the order parameter, as in percolation 
theory. As p increases for a fixed b < bt {bt is defined below), G increases gradually from 
to 1 and exhibits a percolation transition at Pc{b). In Fig. [T](a), we can see that the 
transition interval in which 0.1 < G < 0.4 is wide for small b. However, this interval 
becomes narrower as b increases, indicating that the giant cluster grows suddenly as p 
increases for large values of b. A similar behavior is exhibited by the susceptibility, defined 
by b) = Yl's s^^s, where the prime denotes the exclusion of the giant cluster in the 
summation, and Ug is the normalized number of s-sized clusters composed of permanent 
cooperators. As shown in Fig. [D^b), the peak positions of x mark the phase boundary 
Pc{b) across which the giant cooperator cluster increases to a finite size. The peak height 
of the susceptibility decreases as b approaches the tricritical-like point bt-, beyond which the 
susceptibility is likely to disappear. It is noteworthy that while the peak diverges in the 
classical percolation transition, it is finite in the infinite-order percolation transition, which 
often occurs in growing networks 16|]. On the basis of this observation, we may say that the 
percolation transition is continuous for p < pt and of infinite-order for p > pt. The estimated 
value of Pt is about 0.4; this is roughly equal to p* ~ 0.494, which is the boundary between 
the large-world and the small-world networks as determined on the basis of the thermal 
t..a,.,tio„ patten, of the .ode. Q. 
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FIG. 4: (Color online) Size distribution of cooperator clusters in the hierarchical network with 
N = 10, 924 nodes for (a) various values of p and b = 1.7 and (b) various values of b and p = 0.15. 
The data are obtained by averaging the results for over 500 configurations. 
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FIG. 5: (Color online) Size distribution of cooperator clusters for several b in the the WWW 
with system size N = 325, 729. The data are obtained by averaging the results for over 100 
configurations. 
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The cluster-size distribution Ug is shown in Fig. |H In Fig. Hl^a), n<j is plotted against s for 
several p, but with a fixed h = 1.7 , and in Fig. Ht^b), Us is plotted for several b, but with a 
fixed p = 0.15. For the large-world network, i.e., the network for which p = 0.0 < Pc, nsip) 
exhibits a sub-critical behavior, i.e., for small s, it decays according to a power-law and for 
large s, it decays exponentially beyond a cutoff. At Pc = 0.1, ns{pc) obeys the power law 
UsiPc) ~ s~'^ with r f« 1.85 ± 0.1. When p > pc, ns{p) exhibits super-critical behavior. For 
the small-world network, the cluster size distribution does not follow a power law. Similar 
behavior is observed when p is kept constant while varying h (Fig. ID^b)). When p < pt, the 
order parameter increases gradually with b, while when p > pt, it increases very drastically. 

Large-world networks are known as fractal networks. Typical fractal networks in the real 
world are the protein interaction network and the World-Wide Web (WWW) ll|. In this 
study, we simulate the evolutionary PD game on the WWW. Since the WWW network is 
a single network, corresponding to p being fixed, we vary only b. As shown in Fig. [5l the 
distribution of cooperator cluster sizes for several values of b decays according to a power law 
with an exponent of approximately 2. Thus, we conclude that the critical behavior of the 
cluster size distribution is not limited to the hierarchical networks, but rather, this behavior 
is intrinsic. 

In summary, we have studied the percolation transition of cooperator clusters in artificial 
hierarchical networks as well as in the WWW network. We found that in the WWW, the 
cluster size distribution of permanent cooperators follows a power law near the percolation 
threshold. Such a critical behavior is also observed in the fractal hierarchical networks; on the 
basis of these observation, we can determine the condition under which critical behavior is 
observed. The power-law behavior indicates that cooperators create communities of diverse 
sizes and at scattered locations. These clusters stochastically form affected by hubs. In 
order to improve cooperations on a global scale in society, communication channels must be 
established between influential individuals. 

This study was supported by an NRF grant (Grant No. 2010-0015066 (BK)), by the 
NAP of KRCF (CKY), and by Grants-in-Aid for Scientiflc Research (Nos. 20760258 and 
20540382, and Innovative Areas "Systems Molecular Ethology") from MEXT, Japan (NM). 
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